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Vol. XXXI No. 1 (October 1978), Copyright(c)1978 by Princeton University 
Press, by permission of the copyright holder. 


Military disasters befall some states, no matter how informed their leaders are, 
because their capabilities are deficient. Weakness, not choice, is their primary 
problem. Powerful nations are not immune to calamity either, because their leaders 
may misperceive threats or miscalculate responses. Information, understanding, and 
judgment are a larger part of the strategic challenge for countries such as the United 
States. Optimal decisions in defense policy therefore depend on the use of strategic 
intelligence: the acquisition, analysis, and appreciation of relevant data. In the best- 
known cases of intelligence failure, the most crucial mistakes have seldom been made 
by collectors of raw information, occasionally by professionals who produce finished 
analyses, but most often by the decision makers who consume the products of 
intelligence services. Policy premises constrict perception, and administrative 
workloads constrain reflection. Intelligence failure is political and psychological more 
often than organizational. 

Observers who see notorious intelligence failures as egregious often infer that 
disasters can be avoided by perfecting norms and procedures for analysis and 
argumentation. This belief is illusory. Intelligence can be improved marginally, but 
not radically, by altering the analytic system. The illusion is also dangerous if it abets 
overconfidence that systemic reforms will increase the predictability of threats. The 
use of intelligence depends less on the bureaucracy than on the intellects and 
inclinations of the authorities above it. To clarify the tangled relationship of analysis 
and policy, this essay explores conceptual approaches to intelligence failure, 
differentiation of intelligence problems, insurmountable obstacles to accurate 
assessment, and limitations of solutions proposed by critics. 

I. Approaches to Theory 

Case studies of intelligence failures abound, yet scholars lament the lack of a 
theory of intelligence. 1 It is more accurate to say that we lack a positive or normative 


*For corrections or comments whose usefulness exceeded my ability to accommodate them within 
space limitations, thanks are due to Bruce Blair, Thomas Blau, Michael Handel, Robert Jervis, Klaus Knorr, 
H. R. Trevor-Roper, and members of the staff of the National Foreign Assessment Center. 

‘For example, Klaus Knorr, “Failures in National Intelligence Estimates: The Case of the Cuban 
Missiles,” World Politics , xvi (April 1964), 455, 465-66; Harry Howe Ransom, “Strategic Intelligence and 
Foreign Policy,” World Politics, xxvii (October 1974), 145. 
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theory of intelligence. Negative or descriptive theory — the empirical understanding of 
how intelligence systems make mistakes — is well developed. The distinction is 
significant because there is little evidence that either scholars or practitioners have 
succeeded in translating such knowledge into reforms that measurably reduce failure. 
Development of a normative theory of intelligence has been inhibited because the 
lessons of hindsight do not guarantee improvement in foresight, and hypothetical 
solutions to failure only occasionally produce improvement in practice. The problem 
of intelligence failure can be conceptualized in three overlapping ways. The first is the 
most reassuring; the second is the most common; and the third is the most important. 

1. Failure in perspective. There is an axiom that a pessimist sees a glass of water 
as half empty and an optimist sees it as half full. In this sense, the estimative system is 
a glass half full. Mistakes can happen in any activity. Particular failures are accorded 
disproportionate significance if they are considered in isolation rather than in terms of 
the general ratio of failures to successes; the record of success is less striking because 
observers tend not to notice disasters that do not happen. Any academician who used a 
model that predicted outcomes correctly in four out of five cases would be happy; 
intelligence analysts must use models of their own and should not be blamed for 
missing occasionally. One problem with this benign view is that there are no clear 
indicators of what the ratio of failure to success in intelligence is, or whether many 
successes on minor issues should be reassuring in the face of a smaller number of 
failures on more critical problems. 2 In the thermonuclear age, just one mistake could 
have apocalyptic consequences. 

2. Pathologies of communication. The most frequently noted sources of 
breakdowns in intelligence lie in the process of amassing timely data, communicating 
them to decision makers, and impressing the latter with the validity or relevance of 
the information. This view of the problem leaves room for optimism because it implies 
that procedural curatives can eliminate the dynamics of error. For this reason, official 
post mortems of intelligence blunders inevitably produce recommendations for 
reorganization and changes in operating norms. 

3. Paradoxes of perception. Most pessimistic is the view that the roots of failure 
lie in unresolvable trade-offs and dilemmas. Curing some pathologies with 
organizational reforms often creates new pathologies or resurrects old ones; 3 perfecting 
intelligence production does not necessarily lead to perfecting intelligence consump- 
tion; making warning systems more sensitive reduces the risk of surprise, but increases 
the number of false alarms, which in turn reduces sensitivity; the principles of optimal 
analytic procedure are in many respects incompatible with the imperatives of the 
decision process; avoiding intelligence failure requires the elimination of strategic 
preconceptions, but leaders cannot operate purposefully without some preconceptions. 
In devising measures to improve the intelligence process, policy makers are damned if 
they do and damned if they don’t. 


2 “As that ancient retiree from the Research Department of the British Foreign Office reputedly said, 
after serving from 1903-50: ‘Year after year the worriers and fretters would come to me with awful 
predictions of the outbreak of war. 1 denied it each time. I was only wrong twice.’ ” Thomas L. Hughes, The 
Fate of Facts in a World of Men — Foreign Policy and Intelligence-Making (New York: Foreign Policy 
Association, Headline Series No. 233, December 1976), 48. Paradoxically, “successes may be 
indistinguishable from failures.” If analysts predict war and the attacker cancels his plans because surprise 
has been lost, “success of the intelligence services would have been expressed in the falsification of its 
predictions,” which would discredit the analysis. Avi Shlaim, “Failures in National Intelligence Estimates: 
The Case of the Yom Kippur War,” World Politics, xxvm (April 1976), 378. 

2 Compare the prescriptions in Peter Szanton and Graham Allison, “Intelligence: Seizing the 
Opportunity," with George Carver’s critique, both in Foreign Policy, No. 22 (Spring 1976). 
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It is useful to disaggregate the problem of strategic intelligence failures in order to 
elicit clues about which paradoxes and pathologies are pervasive and therefore most in 
need of attention. The crucial problems of linkage between analysis and strategic 
decision can be subsumed under the following categories: 

1. Attack warning. The problem in this area is timely prediction of an enemy’s 
immediate intentions, and the “selling” of such predictions to responsible authorities. 
Major insights into intelligence failure have emerged from catastrophic surprises: Pearl 
Harbor, the Nazi invasion of the U.S.S.R., the North Korean attack and Chinese 
intervention of 1950, and the 1973 war in the Middle East. Two salient phenomena 
characterize these cases. First, evidence of impending attack was available, but did not 
flow efficiently up the chain of command. Second, the fragmentary indicators of 
alarm that did reach decision makers were dismissed because they contradicted 
strategic estimates or assumptions. In several cases hesitancy in communication and 
disbelief on the part of leaders were reinforced by deceptive enemy maneuvers that 
cast doubt on the data. 4 

2. Operational evaluation. In wartime, the essential problem lies in judging the 
results (and their significance) of interacting capabilities. Once hostilities are under 
way, informed decision making requires assessments of tactical effectiveness — “how 
we are doing” — in order to adapt strategy and options. In this dimension, the most 
interesting insights have come from Vietnam-era memoirs of low-level officials and 
from journalistic muckraking. Again there are two fundamental points. First, within 
the context of a glut of ambiguous data, intelligence officials linked to operational 
agencies (primarily military) tend to indulge a propensity for justifying service 
performance by issuing optimistic assessments, while analysts in autonomous non- 
operational units (primarily in the Central Intelligence Agency and the late Office of 
National Estimates) tend to produce more pessimistic evaluations. Second, in contrast 
to cases of attack warning, fragmentary tactical indicators of success tend to override 
more general and cautious strategic estimates. Confronted by differing analyses, a 
leader mortgaged to his policy tends to resent or dismiss the critical ones, even when 
they represent the majority view of the intelligence community, and to cling to the 
data that support continued commitment. 5 Lyndon Johnson railed at his Director of 


* Roberta Wohlstetter, Pearl Harbor: Warning and Decision (Stanford: Stanford University Press 
1962); Barton Whaley, Codeword Barbarossa (Cambridge: The M.I.T. Press 1973); Harvey De Weerd, 
"Strategic Surprise in the Korean War,” Orbis, vi (Fall 1962); Alan Whiting, China Crosses the Yalu (New 
York: Macmillan 1960); James F. Schnabel, Policy and Direction: The First Year (Washington, D C.: 
Department of the Army 1972), 61-65, 83-85, 274-78; Michael I. Handel, Perception, Deception, and 
Surprise: The Case of the Yom Kippur War (Jerusalem: Leonard Davis Institute of International Relations, 
Jerusalem Paper No. 19, 1976); Shlaim (fn. 2); Abraham Ben-Zvi, “Hindsight and Foresight: A Conceptual 
Framework for the Analysis of Surprise Attacks," World Politics, xxvm (April 1976); Amos Perlmutter, 
“Israel’s Fourth War, October 1973: Political and Military Misperceptions, ’’ Orbis, xtx (Summer 1975); U.S., 
Congress, House, Select Committee on Intelligence [hereafter cited as HSCI], Hearings, U.S. Intelligence 
Agencies and Activities: The Performance of the Intelligence Community, 94th Cong., 1st sess., 1975; 
Draft Report of the House Select Committee on Intelligence, published in The Village Voice, February 16, 
1976, pp. 76-81. 

5 David Halberstam, The Best and the Brightest (New York: Random House 1972); Morris Blachman, 
“The Stupidity of Intelligence,” in Charles Peters and Timothy J. Adams, eds., Inside the System (New 
York: Praeger 1970); Patrick J. McGarvey, “DIA: Intelligence to Please,” in Morton Halperin and Arnold 
Kanter, eds., Readings in American Foreign Policy: A Bureaucratic Perspective (Boston: Little, Brown 
1973); Chester Cooper, “The CIA and Decision-Making,” Foreign Affairs, Vol. 50 (January 1972); Sam 
Adams, “Vietnam Cover-Up: Playing War With Numbers," Harper’s, Vol. 251 (June 1975); Don 
Oberdorfer, Tet! (Garden City, N.Y.: Doubleday 1971). For a more detailed review, see Richard K, Betts, 
Soldiers, Statemen, and Cold War Crises (Cambridge: Harvard University Press 1977), chap. 10. 
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Central Intelligence (DCI) at a White House dinner: “Policy making is like milking a 
fat cow. You see the milk coming out, you press more and the milk bubbles and flows, 
and just as the bucket is full, the cow with its tail whips the bucket and all is spilled. 
That’s what CIA does to policy making. ” 6 From the consensus-seeking politician, this 
was criticism; to a pure analysts, it would have been flattery. But it is the perspective 
of the former, not the latter, that is central in decision making. 

3. Defense planning. The basic task in using intelligence to develop doctrines 
and forces for deterrence and defense is to estimate threats posed by adversaries, in 
terms of both capabilities and intentions, over a period of several years. Here the 
separability of intelligence and policy, analysis and advocacy, is least clear. In dealing 
with the issue of “how much is enough” for security, debates over data merge murkily 
into debates over options and programs. As in operational evaluation, the problem lies 
more in data mongering than in data collecting. To the extent that stark 
generalizations are possible, the basic points in this category are the reverse of those in 
the previous one. 

First, the justification of a mission (in this case, preparedness for future 
contingencies as opposed to demonstration of current success on the battlefield) 
prompts pessimistic estimates by operational military analysts; autonomous analysts 
without budgetary axes to grind, but with biases similar to those prevalent in the 
intellectual community, tend toward less alarmed predictions. 7 Military intelligence 
inclines toward “worst-case” analysis in planning, and toward “best-case” analysis in 
operational evaluation. (Military intelligence officials such as Lieutenant General 
Daniel Graham were castigated by liberals for underestimating the Vietcong’s strength 
in the 1960’s but for overestimating Soviet strength in the 1970’s.) Air Force 
intelligence overestimated Soviet air deployments in the “bomber gap” controversy of 
the 1950’s, and CIA-dominated National Intelligence Estimates (NIE’s) under- 
estimated Soviet ICBM deployments throughout the 1960’s (over-reacting, critics say, 
to the mistaken prediction of a “missile gap” in I960). 8 

Second, in the context of peacetime, with competing domestic claims on 
resources, political leaders have a natural interest in at least partially rejecting military 
estimates and embracing those of other analysts who justify limiting allocations to 
defense programs. If the President had accepted pessimistic CIA operational 
evaluations in the mid-1960’s, he might have withdrawn from Vietnam; if he had 
accepted pessimistic military analyses of the Soviet threat in the mid-1970 s, he might 
have added massive increases to the defense budget. 


6 Quoted in Henry Brandon, The Retreat of American Power (Garden City, N.Y.: Doubleday 1973), 

103. 

' Betts (fn. 5), 160-61, 192-95. On bias within CIA, see James Schlesinger’s comments in U.S., 
Congress, Senate, Select Committee to Study Governmental Operations with Respect to Intelligence 
Activities [hereafter cited as SSCI], Final Report, Foreign and Military Intelligence , Book I, 94th Cong., 2d 
sess., 1976, 76-77. 

8 Ibid., Book IV, 56-59; William T. Lee, Understanding the Soviet Military Threat: How CIA 
Estimates Went Astray (New York: National Strategy Information Center, Agenda Paper No. 6, 1977), 24- 
37; Albert Wohlstetter: “Is There a Strategic Arms Race?” Foreign Policy, No. 15 (Summer 1974); 
Wohlstetter, “Rivals, But No Race,” Foreign Policy, No. 16 (Fall 1974); Wohlstetter, "Optimal Ways to 
Confuse Ourselves,” Foreign Policy, No. 20 (Fall 1975). There are exceptions to this pattern of military and 
civilian bias: see ibid., 185-88; Lieutnant General Daniel Graham, USA (Ret.), “The Intelligence Mythology 
of Washington,” Strategic Review, iv (Summer 1976), 61-62, 64; Victor Marchetti and John Marks, The CIA 
and the Cult of Intelligence (New York: Knopf 1974), 309. 
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Some chronic sources of error are unique to each of these three general categories 
of intelligence problems, and thus do not clearly suggest reforms that would be 
advisable across the board. To compensate for the danger in conventional attack 
warning, reliance on worst-case analysis might seem the safest rule, but in making 
estimates for defense planning, worst-case analysis would mandate severe and often 
unnecessary economic sacrifices. Removing checks on the influence of CIA analysts 
and “community” staffs 9 might seem justified by the record of operational evaluation 
in Vietnam, but would not be warranted by the record of estimates on Soviet ICBM 
deployments. It would be risky to alter the balance of power systematically among 
competing analytic components, giving the “better” analysts more status. Rather, 
decision makers should be encouraged to be more and less skeptical of certain 
agencies’ estimates, depending on the category of analysis involved. 

Some problems, however, cut across all three categories and offer a more general 
basis for considering changes in the system. But these general problems are not very 
susceptible to cure by formal changes in process, because it is usually impossible to 
disentangle intelligence failures from policy failures. Separation of intelligence and 
policy making has long been a normative concern of officials and theorists, who have 
seen both costs and benefits in minimizing the intimacy between intelligence 
professionals and operational authorities. But, although the personnel can be 
segregated, the functions cannot, unless intelligence is defined narrowly as the 
collection of data, and analytic responsibility is reserved to decision makers. Analysis 
and decision are interactive rather than sequential processes. By the narrower 
definition of intelligence, there have actually been few major failures. In most cases of 
mistakes in predicting attacks or in assessing operations, the inadequacy of critical data 
or their submergence in a viscous bureaucracy were at best the proximate causes of 
failure. The ultimate causes of error in most cases have been wishful thinking, cavalier 
disregard of professional analysts, and, above all, the premises and preconceptions of 
policy makers. Fewer fiascoes have occurred in the stages of acquisition and 
presentation of facts than in the stages of interpretation and response. Producers of 
intelligence have been culprits less often than consumers. Policy perspectives tend to 
constrain objectivity, and authorities often fail to use intelligence properly. As former 
State Department intelligence director Ray Cline testified, defending his analysts’ 
performance in October 1973 and criticizing Secretary Kissinger for ignoring them: 
Unless something is totally conclusive, you must make an inconclusive 
report. ... by the time you are sure it is always very close to the event. So I 
don’t think the analysts did such a lousy job. What I think was the lousy job 
> was in bosses not insisting on a new preparation at the end of that week 
[before war broke out], . . . the reason the system wasn’t working very well is 
that people were not asking it to work and not listening when it did work. 10 

II. Basic Barriers to Analytic Accuracy 

Many constraints on the optimal processing of information lie in the structure of 
authority and the allocation of time and resources. Harold Wilensky argues 
persuasively that the intelligence function is hindered most by the structural 


9 The U S. intelligence community includes the CIA, Defense Intelligence Agency (DIA), National 
Security Agency, the intelligence branches of each military service, the State Department Bureau of 
Intelligence and Research, the intelligence units of the Treasury and Energy Departments, and the FBI. 
Before 1973, coordination for national estimates was done through the Office of National Estimates, and 
since then, through the National Intelligence Officers. The Intelligence Community Staff assists the Director 
of Central Intelligence in managing allocation of resources and reviewing the agencies’ performance. 

10 HSCI, Hearings (fn. 4), 656-57. 
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characteristics of hierarchy, centralization, and specialization." Yet it is precisely these 
characteristics that are the essence of any government. A related problem is the 
dominance of operational authorities over intelligence specialists, and the trade-off 
between objectivity and influence. Operators have more influence in decision making 
but are less capable of unbiased interpretation of evidence because they have a vested 
interest in the success of their operations; autonomous analysts are more disinterested 
and usually more objective, but lack influence. Senior generalists at the policy level 
often distrust or discount the judgments of analytic professionals and place more 
weight on reports from operational sources. 12 In response to this phenomenon, the 
suggestion has been made to legislate the requirement that decision makers consider 
analyses by the CIA’s Intelligence Directorate (now the National Foreign Assessment 
Center) before establishing policy. 13 Such a requirement would offer no more than 
wishful formalism. Statutory fiat cannot force human beings to value one source above 
another. “No power has yet been found,” DCI Richard Helms has testified, “to force 
Presidents of the United States to pay attention on a continuing basis to people and 
papers when confidence has been lost in the originator.” 14 Moreover, principals tend 
to believe that they have a wider point of view than middle-level analysts and are 
better able to draw conclusions from raw data. That point of view underlies their 
fascination with current intelligence and their impatience with the reflective 
interpretations in “finished” intelligence. 15 

The dynamics of decision are also not conducive to analytic refinement. In a 
crisis, both data and policy outpace analysis, the ideal process of staffing and 
consultation falls behind the press of events, and careful estimates cannot be digested 
in time. As Winston Churchill recalled of the hectic days of spring 1940, “The 
Defence Committee of the War Cabinet sat almost every day to discuss the reports of 
the Military Co-ordination Committee and those of the Chiefs of Staff; and their 
conclusions or divergences were again referred to frequent Cabinets. All had to be 
explained or reexplained; and by the time this process was completed, the whole scene 
had often changed.” 10 Where there is ample time for decision, on the other hand, the 
previously mentioned bureaucratic impediments gain momentum. 17 Just as informa- 
tion processing is frustrated by constraints on the time that harried principals can 
spend scrutinizing analytic papers, it is constrained by the funds that a government 
can spend. To which priorities should scarce resources be allocated? The Schlesinger 
Report of 1971, which led to President Nixon’s reorganization of U.S. intelligence, 
noted that criticisms of analytic products were often translated into demands for more 

11 Wilensky, Organizational Intelligence (New York: Basic Books 1967), 42-62, 126, 179. 

12 Ibid., passim. The counterpoint of Cooper (fn. 5) and McGarvey (fn. 5) presents a perfect 
illustration. 

13 Graham Allison and Peter Szanton, Remaking Foreign Policy: The Organizational Connection 
(New York: Basic Books 1976), 204. 

14 Quoted in SSCI, Final Report (fn. 7), I, 82. 

15 Ibid., 267, 276; SSCI, Staff Report , Covert Action in Chile 1963-1973, 94th Cong., 1st sess., 1975, 
48-49. The Senate Committee deplored the tendency of decision makers to focus on the latest raw data 
rather than on refined analyses, a practice that contributed to the intelligence failure in the 1974 Cyprus 
crisis. SSCI, Final Report (fn. 7), I, 443. But the failure in the October War was largely due to the reverse 
phenomenon: disregarding warning indicators because they contradicted finished intelligence that 
minimized the possibility of war. HSCI Draft Report (fn. 4), 78; Ben-Zvi (fn. 4), 386, 394; Perlmutter (fn. 4), 
453. 

ir ’ Churchill, The Gathering Storm (Boston: Houghton Mifflin 1948), 587-88. 

17 “Where the end is knowledge, as in the scientific community, time serves intelligence; where the 
end is something else — as in practically every organization but those devoted entirely to scholarship — time 
subverts intelligence, since in the long run, the central institutionalized structures and aims (the 
maintenance of authority, the accommodation of departmental rivalries, the service of established doctrine) 
will prevail.” Wilensky (fn. 11), 77. 
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extensive collection of data, but “Seldom does anyone ask if a further reduction in 
uncertainty, however small, is worth its cost.” 18 Authorities do not always know, 
however, which issues require the greatest attention and which uncertainties harbor 
the fewest potential threats. Beyond the barriers that authority, organization, and 
scarcity pose to intelligence lie more fundamental and less remediable intellectual 
sources of error. 

1. Ambiguity of evidence. Intelligence veterans have noted that “estimating is 
what you do when you do not know,” 19 but “it is inherent in a great many situations 
that after reading the estimate, you will still not know.” 20 These observations highlight 
an obvious but most important obstacle to accuracy in analysis. It is the role of 
intelligence to extract certainty from uncertainty and to facilitate coherent decision in 
an incoherent environment. (In a certain and coherent environment there is less need 
for intelligence.) To the degree they reduce uncertainty by extrapolating from 
evidence riddled with ambiguities, analysts risk oversimplifying reality and 
desensitizing the consumers of intelligence to the dangers that lurk within the 
ambiguities; to the degree they do not resolve ambiguities, analysts risk being 
dismissed by annoyed consumers who see them as not having done their job. 
Uncertainty reflects inadequacy of data, which is usually assumed to mean lack of 
information. But ambiguity can also be aggravated by an excess of data. In attack 
warning, there is the problem of “noise” and deception; in operational evaluation 
(particularly in a war such as Vietnam), there is the problem of overload from the high 
volume of finished analyses, battlefield statistics, reports, bulletins, reconnaissance, 
and communications intercepts flowing upward through multiple channels at a rate 
exceeding the capacity of officials to absorb or scruitinize them judiciously. (From the 
CIA alone, the White House received current intelligence dailies, Weekly Reports, 
daily Intelligence Information Cables, occasional Special Reports and specific 
memoranda, and analyses from the CIA Vietnam Working Group.) Similarly, in 
estimates for defense planning, there is the problem of innumerable and endlessly 
refined indices of the strategic balance, and the dependence of assessments of 
capabilities on complex and variable assumptions about the doctrine, scenarios, and 
intentions that would govern their use. 

Because it is the job of decision makers to decide, they cannot react to ambiguity 
by deferring judgment. 21 When the problem is an environment that lacks clarity, an 
overload of conflicting data, and lack of time for rigorous assessment of sources and 
validity, ambiguity abets instinct and allows intuition to drive analysis. Intelligence 
can fail because the data are too permissive for policy judgment rather than too 
constraining. When a welter of fragmentary evidence offers support to various 
interpretations, ambiguity is exploited by wishfulness. The greater the ambiguity, the 
greater the impact of preconceptions. 22 (This point should be distinguished from the 

Quoted in SSCI, Final Report (fn. 7), I, 274. 

19 Sherman Kent, “Estimates and Influence," Foreign Service Journal , xi.vi (April 1969), 17. 

20 Hughes (fn. 2), 43. 

21 "The textbooks agree, of course, that we should only believe reliable intelligence, and should never 
cease to be suspicious, but what is the use of such feeble maxims? They belong to that wisdom which for 
want of anything better scribblers of systems and compendia resort to when they run out of ideas.” Carl von 
Clausewitz, On War, ed. and trans. by Michael Howard and Peter Paret (Princeton: Princeton University 
Press 1976), 117. 

22 Robert Jervis, The Logic of Images in International Relations (Princeton: Princeton University 
Press 1970), 132; Jervis, Perception and Misperception in International Politics (Princeton: Princeton 
University Press 1976), chap. 4; Floyd Allport, Theories of Perception and the Concept of Structure, cited 
in Shlaim (fn. 2), 358. Cognitive theory suggests that uncertainty provokes decision makers to separate 
rather than integrate their values, to deny that inconsistencies between values exist, and even to see 
contradictory values as mutually supportive. John Steinbruner, The Cybernetic Theory of Decision 
(Princeton: Princeton University Press 1974), 105-8. 
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theory of cognitive dissonance, which became popular with political scientists at the 
time it was being rejected by psychologists.) 23 There is some inverse relation between 
the importance of an assessment (when uncertainty is high) and the likelihood that it 
will be accurate. Lyndon Johnson could reject pessimistic NIE’s on Vietnam by 
inferring more optimistic conclusions from the reports that came through command 
channels on pacification, interdiction, enemy casualties, and defections. Observers 
who assume Soviet malevolence focus on analyses of strategic forces that emphasize 
missile throw-weight and gross megatonnage (Soviet advantages); those who assume 
more benign Soviet intentions focus on analyses that emphasize missile accuracy and 
numbers of warheads (U.S. advantages). In assessing the naval balance, Secretary of 
Defense Rumsfeld focused on numbers of ships (Soviet lead), and Congressman Les 
Aspin, a critic of the Pentagon, focused on total tonnage (U.S. lead). 

2. Ambivalence of judgment. Where there are ambiguous and conflicting 
indicators (the context of most failures of intelligence), the imperatives of honesty and 
accuracy leave a careful analyst no alternative but ambivalence. There is usually some 
evidence to support any prediction. For instance, the CIA reported in June 1964 that a 
Chinese instructor (deemed not “particularly qualified to make this remark”) had told 
troops in a course in guerrilla warfare, “We will have the atom bomb in a matter of 
months.” 24 Several months later the Chinese did perform their first nuclear test. If the 
report had been the only evidence, should analysts have predicted the event? If they 
are not to make a leap of faith and ignore the data that do not mesh, analysts will issue 
estimates that waffle. In trying to elicit nuances of probability from the various 
possibilities not foreclosed by the data, cautious estimates may reduce ambivalence, 
but they may become Delphic or generalized to the point that they are not useful 
guides to decision. (A complaint I have heard in conversations with several U.S. 
officials is that many past estimates of Soviet objectives could substitute the name of 
any other great power in history — Imperial Rome, 16th-century Spain, Napoleonic 
France — and sound equally valid.) Hedging is the legitimate intellectual response to 
ambiguity, but it can be politically counterproductive, if the value of intelligence is to 
shock consumers out of wishfulness and cognitive insensitivity. A wishful decision 
maker can fasten onto that half of an ambivalent analysis that supports his 
predisposition. 25 A more objective official may escape this temptation, but may 
consider the estimate useless because it does not provide “the answer.” 

3. Atrophy of reforms. Disasters always stimulate organizational changes 
designed to avert the same failures in the future. In some cases these changes work. In 
many instances, however, the changes persist formally but erode substantively. 
Standard procedures are constant. Dramatic failures occur only intermittently. If the 
reforms in procedure they have provoked do not fulfill day-to-day organizational 
needs — or if, as often happens, they complicate operations and strain the 
organization’s resources — they fall into disuse or become token practices. After the 

23 See William J. McGuire, “Selective Exposure: A Summing Up,” in R. P. Abelson and others, eds., 
Theories of Cognitive Consistency (Chicago: Rand McNally 1968), and Irving L. Janis and Leon Mann, 
Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment (New York: Free Press 
1977), 213-14. 

1 CIA Intelligence Information Cable, “Remarks of the Chief of the Nanking Military Academy and 
Other Chinese Leaders on the Situation in South Vietnam,” June 25, 1964, in Lyndon B. Johnson Library 
National Security Files, Vietnam Country File [hereafter cited as LBJL/NSF-VNCF], Vol. XII, item 55. 

25 See for example, U.S., Department of Defense, The Senator Gravel Edition: The Pentagon Papers 
(Boston: Beacon Press 1971) [hereafter cited as Pentagon Papers ], Vol. II, 99; Frances Fitzgerald, Fire in the 
Lake (Boston: Atlantic-Little, Brown 1972), 364; Special National Intelligence Estimate 53-64, “Chances for 
a Stable Government in South Vietnam,” September 18, 1964, and McGeorge Bundy’s covering letter to the 
President, in LBJL/NSF-VNCF, Vol. XIII, item 48. 
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postmortem of North Korea’s downing of a U.S. EC-121 monitoring aircraft in 1969, 
there was, for several months, a great emphasis on risk assessments for intelligence 
collection missions. Generals and admirals personally oversaw the implementaton of 
new procedures for making the assessments. Six months later, majors and captains 
were doing the checking. “Within a year the paperwork was spot-checked by a major 
and the entire community slid back to its old way of making a ‘quick and dirty’ 
rundown of the JCS criteria when sending in reconnaissance mission proposals. ” 26 The 
downing of the U-2 over the Soviet Union in 1960 and the capture of the intelligence 
ship Pueblo in 1968 had been due in part to the fact that the process of risk assessment 
for specific collection missions, primarily the responsibility of overworked middle- 
level officers, had become ponderous, sloppy, or ritualized. 27 At a higher level, a 
National Security Council Intelligence Committee was established in 1971 to improve 
responsiveness of intelligence staff to the needs of policy makers. But since the 
subcabinet-level consumers who made up the committee were pressed by other 
responsibilities, it lapsed in importance and was eventually abolished. 28 A comparable 
NSC committee that did serve tangible day-to-day needs of consumers to integrate 
intelligence and policy — the Verification Panel, which dealt with SALT — was more 
effective, but it was issue-oriented rather than designed to oversee the intelligence 
process itself. Organizational innovations will not improve the role of intelligence in 
policy unless they flow from the decision makers’ views of their own needs and unless 
they provide frequent practical benefits. 

None of these three barriers are accidents of structure or process. They are 
inherent in the nature of intelligence and the dynamics of work. As such, they 
constitute severe constraints on the efficacy of structural reform. 

III. The Elusiveness of Solutions 

If they do not atrophy, most solutions proposed to obviate intelligence 
dysfunctons have two edges: in reducing one vulnerability, they increase another. 
After the seizure of the Pueblo, the Defense Intelligence Agency (DIA) was 
reprimanded for misplacing a message that could have prevented the incident. The 
colonel responsible developed a careful microfilming operation in the message center 
to ensure a record of transmittal of cables to authorities in the Pentagon. 
Implementing this check, however, created a three-to-four hour delay — another 
potential source of failure — in getting cables to desk analysts whose job was to keep 
reporting current. 29 Thus, procedural solutions often constitute two steps forward and 
one step back; organizational fixes cannot transcend the basic barriers. The lessons of 
Pearl Harbor led to the establishment of a Watch Committee and National Indications 
Center in Washington. Although this solution eliminated a barrier in the 
communication system, it did not prevent the failure of timely alert to the Chinese 
intervention in Korea or the 1973 October War, because it did not eliminate the 
ambiguity barrier. (Since then, the Watch Committee has been replaced by the DCI’s 


” Patrick J. McGarvey, CIA: The Myth and the Madness (Baltimore: Penguin 1974), 16. 

27 David Wise and Thomas B. Boss, The U-2 Affair (New York: Random House 1962), 56, 176, 180; 
Trevor Armbrister, A Matter of Accountability (New York: Coward-McCann 1970), 116-18, 141-45, 159, 
187-95; U.S., Congress, House, Committee on Armed Services, Report, Inquiry Into the U.S.S. Pueblo and 
EC-121 Plane Incidents [hereafter cited as Pueblo and EC-121 Report ], 91st Cong., 1st sess., 1969, 1622-24, 
1650-51; U.S., Congress, House, Committee on Armed Services, Hearings, Inquiry Into the U.S.S. Pueblo 
and EC-121 Plane Incidents [hereafter cited as Pueblo and EC-121 Hearings ], 91st Cong., 1st sess., 1969, 
693-94, 699-700, 703-7, 714, 722, 734, 760, 773-78, 815-16. 

2 »SSCI, Final Report (fn. 7), I, 61-62; HSCI Draft Report (fn. 4), 82. 

22 McGarvey (fn. 26), 16. 
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Strategic Warning Staff.) DIA was reorganized four times within its first ten years; yet 
it continued to leave most observers dissatisfied. The Agranat Commission s review of 
Israel’s 1973 intelligence failure produced proposals for institutional reform that are 
striking because they amount to copying the American system of the same time — 
which had failed in exactly the same way as the Israeli system. 30 Reform is not 
hopeless, but hopes placed in solutions most often proposed — such as the following — 
should be circumscribed. 

1. Assume the worst. A common reaction to traumatic surprise is the 
recommendation to cope with ambiguity and ambivalence by acting on the most 
threatening possible interpretations. If there is any evidence of threat, assume it is 
valid, even if the apparent weight of contrary indicators is greater. In retrospect, when 
the point of reference is an actual disaster attributable to a mistaken calculation of 
probabilities, this response is always justifiable, but it is impractical as a guide to 
standard procedure. Operationalizing worst-case analysis requires extraordinary 
expense; it risks being counterproductive if it is effective (by provoking enemy 
countermeasures or pre-emption), and it is likely to be ineffective because 
routinization will discredit it. Many Israeli observers deduced from the 1973 surprise 
that defense planning could only rest on the assumption that no attack warning will be 
available, and that precautionary mobilization should always be undertaken even 
when there is only dubious evidence of impending Arab action. 31 Similarly, American 
hawks argue that if the Soviets’ intentions are uncertain, the only prudent course is to 
assume they are seeking the capability to win a nuclear war. 

In either case, the norm of assuming the worst poses high financial costs. 
Frequent mobilizations strain the already taut Israeli economy. Moreover, countermo- 
bilization can defeat itself. Between 1971 and 1973, the Egyptians three times 
undertook exercises similar to those that led to the October attack; Israel mobilized in 
response, and nothing happened. It was the paradox of self-negating prophecy. 32 The 
Israeli Chief of Staff was sharply criticized for the unnecessary cost. 33 The danger of 
hypersensitivity appeared in 1977, when General Gur believed Sadat’s offer to come 
to Jerusalem to be a camouflage for an Egyptian attack; he began Israeli maneuvers in 
the Sinai, which led Egypt to begin maneuvers of its own, heightening the risk of 
accidental war. 34 To estimate the requirements for deterrence and defense, worst-case 
assumptions present an open-ended criterion. The procurement of all the hedges 
possible for nuclear war-fighting — large increments in offensive forces, alert status, 
hardening of command-control-and-communications, active and passive defenses — 


30 Shlaim (fn. 2), 375-77. The proposals follow, with their U.S. analogues noted in parentheses: appoint 
a special intelligence adviser to the Prime Minister (Director of Central Intelligence) to supplement the 
military chief of intelligence; reinforce the Foreign Ministry’s research department (Bureau of Intelligence 
and Research); more autonomy for non-military intelligence (CIA); amend rules for transmitting raw 
intelligence to research agencies, the Defense Minister, and the Prime Minister (routing of signals 
intelligence from the National Security Agency); restructure military intelligence (creation of DIA in 1961); 
establish a central evaluation unit (Office of National Estimates). On the U.S. intelligence failure in 1973, 
see the HSCI Draft Report (fn. 4), 78-79. 

31 Shlaim (fn. 2), 379; Handel (fn. 4), 62-63. 

33 Ibid., 55. 

33 Shlaim (fn. 2), 358-59. The Israeli command estimated a higher probability of attack in May 1973 
than it did in October. Having been proved wrong in May, Chief of Staff Elazar lost credibility in 
challenging intelligence officers, complained that he could no longer argue effectively against them, and 
consequently was unable to influence his colleagues when he was right. Personal communication from 
Michael Handel, November 15, 1977. 

34 Washington Post, November 27, 1977, p. A17. 
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would add billions to the U.S. defense budget. Moreover, prudent hedging in policy 
should be distinguished from net judgment of probabilities in estimates. 35 

Alternatively, precautionary escalation or procurement may act as self-fulfilling 
prophecies, either through a catalytic spiral of mobilization (a la World War I) or an 
arms race that heightens tension, or doctrinal hedges that make the prospect of nuclear 
war more “thinkable.” Since evidence for the “action-reaction” hypothesis of U.S. and 
Soviet nuclear policies is meager, and arms races can sometimes be stabilizing rather 
than dangerous, the last point is debatable. Still, a large unilateral increase in strategic 
forces by either the United States or the Soviet Union would, at the least, destroy the 
possibility of gains desired from SALT. A surprise attack or defeat make the costs of 
underestimates obvious and dramatic; the unnecessary defense costs due to 
overestimates can only be surmised, since the minimum needed for deterrence is 
uncertain. Worst-case analysis as a standard norm would also exacerbate the “cry 
wolf” syndrome. Unambiguous threat is not an intelligence problem; rather, the 
challenge lies in the response to fragmentary, contradictory, and dubious indicators. 
Most such indicators turn out to be false alarms. Analysts who reflexively warn of 
disaster are soon derided as hysterical. General William Westmoreland recalled that 
the warnings that had been issued before the 1968 Tet Offensive were ignored. U.S. 
headquarters in Saigon had each year predicted a winter-spring offensive, “and every 
year it had come off without any dire results. ... Was not the new offensive to be 
more of the same?” 3r ’ 

Given the experience of intelligence professionals that most peacetime indicators 
of suspicious enemy activity lead to nothing, what colonel who has the watch some 
night will risk “lighting up the board” in the White House simply on the basis of weak 
apprehension? How many staffers will risk waking a tired President, especially if they 
have done so before and found the action to be needless? How many distracting false 
alarms will an overworked President tolerate before he makes it clear that aides should 
exercise discretion in bothering him? Even if worst-case analysis is promulgated in 
principle, it will be compromised in practice. Routinization corrodes sensitivity. Every 
day that an expected threat does not materialize dulls receptivity to the reality of 
danger. As Roberta Wohlstetter wrote of pre-Pearl Harbor vigilance, “We are 
constantly confronted by the paradox of pessimistic realism of phrase coupled with 
loose optimism in practice.” 37 Seeking to cover all contingencies, worst-case analysis 
loses focus and salience; by providing a theoretical guide for everything, it provides a 
practical guide for very little. 

2. Multiple advocacy. Blunders are often attributed to decision makers’ 
inattention to unpopular viewpoints or to a lack of access to higher levels of authority 
by dissident analysts. To reduce the chances of such mistakes, Alexander George 
proposes institutionalizing a balanced, open, and managed process of debate, so that no 
relevant assessments will be submerged by unchallenged premises or the bureaucratic 
strength of opposing officials. 38 The goal is unobjectionable, and formalized multiple 
advocacy certainly would help, not hinder. But confidence that it will help 
systematically and substantially should be tentative. In a loose sense, there has usually 

35 Raymond Garthoff, “On Estimating and Imputing Intentions,” International Security, n (Winter 
1978), 22. 

36 Westmoreland, A Soldier Reports (Garden City, N.Y.: Doubleday 1976), 316. See the postmortem 
by the President’s Foreign Intelligence Advisory Board, *quoted in Herbert Y. Schandler, The Unmaking of 
a President (Princeton: Princeton University Press 1977), 70, 76, 79-80. 

37 Wohlstetter (fn. 4), 69. 

38 George, “The Case for Multiple Advocacy in Making Foreign Policy,” American Political Science 
Review, Vol. 66 (September 1972). My usage of the term multiple advocacy is looser than George’s. 
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been multiple advocacy in the U.S. policy process, but it has not prevented mistakes in 
deliberation or decision, Lyndon Johnson did not decide for limited bombing and 
gradual troop commitment in Vietnam in 1965 because he was not presented with 
extensive and vigorous counterarguments. He considered seriously (indeed solicited) 
Under Secretary of State George Ball's analysis, which drew on NIE’s and lower-level 
officials’ pessimistic assessments that any escalation would be a mistake. Johnson was 
also well aware of the arguments by DCI John McCone and the Air Force from the 
other extreme — that massive escalation in the air war was necessary because 
gradualism would be ineffective. 39 The President simply chose to accept the views of 
the middle-of-the-road opponents of both Ball and McCone. 

To the extent that multiple advocacy works, and succeeds in maximizing the 
number of views promulgated and in supporting the argumentive resources of all 
contending analysts, it may simply highlight ambiguity rather than resolve it. In 
George’s ideal situation, the process would winnow out unsubstantiated premises and 
assumptions about ends-means linkages. But in the context of data overload, 
uncertainty, and time constraints, multiple advocacy may in effect give all of the 
various viewpoints an aura of empirical respectability and allow a leader to choose 
whichever accords with his predisposition. 40 The efficacy of multiple advocacy (which 
is greatest under conditions of manageable data and low ambiguity) may vary 
inversely with the potential for intelligence failure (which is greatest under conditions 
of confusing data and high uncertainty). The process could, of course, bring to the 
surface ambiguities where false certainty had prevailed; in these cases, it would be as 
valuable as George believes. But if multiple advocacy increases ambivalence and 
leaders do not indulge their instincts, it risks promoting conservatism or paralysis. 
Dean Aeheson saw danger in presidential indecisiveness aggravated by debate: “ ‘I 
know your theory,’ he grumbled to Neustadt. ‘You think Presidents should be warned. 
You’re wrong. Presidents should be given confidence ’’ 41 Even Clausewitz argued 
that deference to intelligence can frustrate bold initiative and squander crucial 
opportunities. Critics charged Henry Kissinger with crippling U.S. intelligence by 
refusing to keep analysts informed of his intimate conversations with foreign leaders. 42 
To do so, however, would have created the possibility of leaks and might thereby have 
crippled his diplomatic maneuvers. It is doubtful that Nixon’s initiative to China could 
have survived prior debate, dissent, and analysis by the bureaucracy. 

It is unclear that managed multiple advocacy would yield markedly greater 
benefits than the redundancy and competitiveness that have long existed. (At best it 
would perfect the “market’ of ideas in the manner that John Stuart Mill believed 


39 Henry F. Graff, The Tuesday Cabinet (Englewood Cliffs, N.J.: Prentice-Hall 1970), 68-71; Leslie 
H. Gelb with Richard K. Betts, The Irony of Vietnam: The System Worked (Washington, DC.: Brookings, 
forthcoming), chap. 4; Ball memorandum of October 5, 1964, reprinted as “Top Secret. The Prophecy the 
President Rejected,” Atlantic Monthly, Vol. 230 (July 1972); McCone, memorandum of April 2, 1965, in 
LBJL/NSF-VNCF, Troop Decision folder, item 14b. 

10 Betts (fn. 5), 199-202; Schandler (fn. 36), 177. George (fn. 38), 759, stipulates that multiple advocacy 
requires “no major maldistribution” of power, influence, competence, information, analytic resources, and 
bargaining skills. But, except for resources and the right to representation, the foregoing are subjective 
factors that can rarely be equalized by design. If they are equalized, in the context of imperfect data and 
time pressure, erroneous arguments as well as accurate ones will be reinforced. Non-expert principals have 
difficulty arbitrating intellectually between experts who disagree. 

41 Quoted in Steinbruner (fn, 22), 332. 

42 Clausewitz (fn. 21), 117-18; HSC1 Hearings (fn. 4), 634-36; William J. Barnds, “Intelligence and 
Policymaking in an Institutional Context,” in U.S., Commission on the Organization of the Government for 
the Conduct of Foreign Policy [hereafter cited as Murphy Commission], Appendices (Washington, D.C.: 
G.P.O., June 1975), Vol. VII, 32. 
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made liberalism conducive to the emergence of truth.) The first major reorganization 
of the American intelligence community in 1946-1947 emphasized centralization in 
order to avert future Pearl Harbors caused by fragmentation of authority; the latest 
reorganization (Carter’s 1977 extension of authority of the Director of Central 
Intelligence over military intelligence programs) emphasized centralization to 
improve efficiency and coherence. Yet decentralization has always persisted in the 
overlapping division of labor between several separate agencies. Recent theorists of 
bureaucracy see such duplication as beneficial because competition exposes 
disagreement and presents policy makers with a wider range of views. Redundancy 
inhibits consensus, impedes the herd instinct in the decision process, and thus reduces 
the likelihood of failure due to unchallenged premises or cognitive errors. To ensure 
that redundancy works in this way, critics oppose a process that yields coordinated 
estimates — negotiated to the least common denominator, and cleared by all agencies 
before they are passed to the principals. George’s “custodian” of multiple advocacy 
could ensure that this does not happen. There are, of course, trade-off costs for 
redundancy. Maximization of competition limits specialization. In explaining the 
failure of intelligence to predict the 1974 coup in Portugal, William Hyland pointed 
out, “if each of the major analytical components stretch their resources over the same 
range, there is the risk that areas of less priority will be superficially covered.”'' 3 

The problem with arguing that the principals themselves should scrutinize 
numerous contrasting estimates in their integrity is that they are constantly 
overwhelmed by administrative responsibilities and “action items”; they lack the time 
to read, ponder, and digest that large an amount of material. Most intelligence 
products, even NIE’s, are never read by policy makers; at best, they are used by 
second-level staffers as background material for briefing their seniors. 44 Consumers 
want previously coordinated analyses in order to save time and effort. In this respect, 
the practical imperatives of day-to-day decision contradict the theoretical logic of 
ideal intelligence. 

3. Consolidation. According to the logic of estimative redundancy, more analysis 
is better than less. Along this line of reasoning. Senate investigators noted critically 
that, as of fiscal year 1975, the U.S. intelligence community still allocated 72 percent 
of its budget for collection of information, 19 percent for processing technical data, 
and less than 9 percent for production of finished analyses. On the other hand 
according to the logic of those who focus on the time constraints of leaders and the 
confusion that results from innumerable publications, quantity counteracts quality. 
The size of the CIA’s intelligence directorate and the complexity of the production 
process “precluded close association between policymakers and analysts, between the 
intelligence product and policy informed by intelligence analysis.” 45 For the sake of 
clarity and acuity, the intelligence bureaucracy should be streamlined. 

This view is consistent with the development of the Office of National Estimates 
(ONE), which was established in 1950 and designed to coordinate the contributions of 
the various organs in the intelligence community for the Director of Central 
Intelligence. DCI Walter Bedell Smith envisioned an operation of about a thousand 
people. But William L. Langer, the scholar Smith imported to organize ONE, wanted 


"HSCI, Hearings (fn. 4), 778. 

44 SSCI, Final Report (fn. 7), IV, 57; Roger Hilsman, Strategic Intelligence and National Decisions 
(Glencoe, 111.: Free Press 1956), 40. During brief service as just a low-level staff member of the National 
Security Council, even I never had time to read all the intelligence analyses relevant to my work. 

45 SSCI, Final Report (fn. 7), I, 344, and IV, 95 (emphasis deleted). 
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a tight group of excellent analysts and a personnel ceiling of fifty. Langer prevailed, 
and though the number of staff members in ONE crept upwards, it probably never 
exceeded a hundred in its two decades of existence. 46 Yet ONE could not eliminate the 
complexity of the intelligence process; it could only coordinate and integrate it for the 
production of National Intelligence Estimates. Other sources found conduits to 
decision makers (to Cabinet members through their own agencies, or to the President 
through the National Security Council). And some policy makers, though they might 
dislike the cacophony of multiple intelligence agencies, were suspicious of the 
consolidated NIE’s, knowing that there was pressure to compromise views in order to 
gain agreement. Over time, the dynamics of bureaucracy also blunted the original 
objectives of ONE’s founder. From a cosmopolitan elite corps, it evolved into an 
insular unit of senior careerists from the CIA. The National Intelligence Officer 
system that replaced ONE reduced the number of personnel responsible for 
coordinating NIE’s, but has been criticized on other grounds such as greater 
vulnerability to departmental pressures. Bureaucratic realities have frustrated other 
attempts to consolidate the intelligence structure. The Defense Intelligence Agency 
was created in 1961 to unify Pentagon intelligence and reduce duplicative activities of 
the three service intelligence agencies, but these agencies regenerated themselves; in 
less than a decade they were larger than they had been before DIA’s inception. 47 

The numerous attempts to simplify the organization of the analytic process thus 
have not solved the major problems. Either the streamlining exercises were short-lived, 
and bureaucratization crept back, or the changes had to be moderated to avoid the 
new dangers they entailed. Contraction is inconsistent with the desire to minimize 
failure by “plugging holes” in intelligence, since compensating for an inadequacy 
usually requires adding personnel and mechanisms; pruning the structure that 
contributes to procedural sluggishness or complexity may create lacunae in substantive 
coverage. 

4. Devil’s advocacy. Multiple advocacy ensures that all views held by individuals 
within the analytic system will be granted serious attention. Some views that should 
receive attention, however, may not be held by anyone within the system. Virtually no 
analysts in Israel or the United States believed the Arabs would be “foolish” enough to 
attack in 1973. Many observers have recommended institutionalizing dissent by 
assigning to someone the job of articulating apparently ridiculous interpretations to 
ensure that they are forced into consideration. Establishing an official devil’s advocate 
would probably do no harm (although some argue that it may preversely facilitate 
consensus-building by domesticating true dissenters or providing the illusory comfort 
that all views have been carefully examined; 48 worse, it might delude decision makers 
into believing that uncertainties have been resolved). But in any case, the role is likely 
to atrophy into a superfluous or artificial ritual. By the definition of the job, the devil’s 
advocate is likely to be dismissed by decision makers as a sophist who only makes an 
argument because he is supposed to, not because of its real merits. Institutionalizing 
devil’s advocacy is likely to be perceived in practice as institutionalizing the “cry 
wolf” problem; “There are limits to the utility of a 'devil s advocate’ who is not a true 
devil.” 49 He becomes someone to be indulged and disregarded. Given its rather sterile 


48 Ray S. Cline, Secrets, Spies, and Scholars (Washington, D.C.: Acropolis 1976), 20. 

47 Gilbert W. Fitzhugh and others, Report to the President and the Secretary of Defense on the 
Department of Defense, By the Blue Ribbon Defense Panel (Washington, D C.: G.P.O., July 1970), 45-46. 

48 Alexander George, “The Devil’s Advocate: Uses and Limitations,” Murphy Commission, 
Appendices (fn. 42), II, 84-85; Jervis, Perception and Misperception (fn. 22), 417. 

n lbid„ 416. 
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definition, the role is not likely to be filled by a prestigious official (who will prefer 
more “genuine” responsibility); it will therefore be easier for policy makers to dismiss 
the arguments. In order to avert intelligence failures, an analyst is needed who tells 
decision makers what they don’t want to hear, dampening the penchant for wishful 
thinking. But since it is the job of the devil’s advocate to do this habitually, and since 
he is most often wrong (as would be inevitable, since otherwise the conventional 
wisdom would eventually change), he digs his own grave. If the role is routinized and 
thus ritualized, it loses impact; but if it is not routinized, there can be no assurance that 
it will be operating when it is needed. 

Despite the last point, which is more important in attack warning than in 
operational evaluation or defense planning, there is a compromise that offers more 
realistic benefits: ad hoc utilization of “real devils.” This selective or biased form of 
multiple advocacy may be achieved by periodically giving a platform within the 
intelligence process to minority views that can be argued more persuasively by 
prestigious analysts outside the bureaucracy. This is what the President’s Foreign 
Intelligence Advisory Board and DCI George Bush did in 1976 by commissioning the 
“Team B” critique of NIE’s on Soviet strategic objectives and capabilities. Dissenters 
within the intelligence community who were skeptical of Soviet intentions were 
reinforced by a panel of sympathetic scholars, with a mandate to produce an analysis 
of their own. 50 This controversial exercise, even if it erred in many of its own ways (as 
dovish critics contend), had a major impact in promoting the re-examination of 
premises and methodology in U.S. strategic estimates. The problem with this option is 
that it depends on the political biases of the authorities who commission it. If it were 
balanced by a comparable “Team C” of analysts at the opposite extreme (more 
optimistic about Soviet intentions than the intelligence community consensus), the 
exercise would approach regular multiple advocacy, with the attendant limitations of 
that solution. Another variant would be intermittent designation of devil's advocates in 
periods of crisis, when the possibility of disaster is greater than usual. Since the role 
would then be fresh each time, rather than ritualized, the advocate might receive a 
more serious hearing. The problem here is that receptivity of decision makers to 
information that contradicts preconceptions varies inversely with their personal 
commitments, and commitments grow as crisis progresses. 51 

5. Sanctions and incentives. Some critics attribute intelligence failures to 
dishonest reporting or the intellectual mediocrity of analysts. Suggested remedies 
include threats of punishment for the former, and inducements to attract talent to 
replace the latter. Other critics emphasize that, will or ability aside, analytic integrity 
is often submerged by the policy makers’ demands for intelligence that suits them; 
“the NIEs ought to be responsive to the evidence, not the policymaker.” 52 Holders of 
this point of view would institutionalize the analysts’ autonomy. Unobjectionable in 
principle (though if analysts are totally unresponsive to the consumer, he will ignore 
them), these implications cannot easily be operationalized without creating as many 
problems as they solve. 

Self-serving operational evaluations from military sources, such as optimistic 
reports on progress in the field in Vietnam or pessimistic strategic estimates, might 


50 U.S., Congress, Senate, Select Committee on Intelligence, Report, The National Intelligence 
Estimates A-B Team Episode Concerning Soviet Capability and Objectives, 95th Cong. 2d sess.. 1978; 
New York Times, December 26, 1976, pp. 1, 14; Washington Post, January 2, 1977, pp. Al, A4. 

51 George H. Poteat, "The Intelligence Gap: Hypotheses on the Process of Surprise,” International 
Studies Notes, 111 (Fall 1976), 15. 

“Cline (fn. 46), 140. 
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indeed be obviated if analysts in DIA, the service intelligence agencies, and command 
staffs were credibly threatened with sanctions (firing, nonpromotion, reprimand, or 
disgrace). Such threats theoretically could be a countervailing pressure to the career 
incentives analysts have to promote the interests of their services. But, except in the 
most egregious cases, it is difficult to apply such standards without arbitrariness and 
bias, given the problem of ambiguity; it simply encourages an alternative bias or 
greater ambivalence. Moreover, military professionals would be in an untenable 
position, pulled in opposite directions by two sets of authorities. To apply the 
sanctions, civil authorities would have to violate the most hallowed military canon by 
having civilian intelligence officials interfere in the chain of command. In view of 
these dilemmas, it is easier to rely on the limited effectiveness of redundancy or 
multiple advocacy to counteract biased estimates. 

Critics concerned with attracting better talent into the analytic bureaucracy 
propose to raise salaries and to provide more high-ranking positions (supergrades) to 
which analysts can aspire. Yet American government salaries are already very high by 
academic standards. Those who attribute DIA’s mediocrity (compared to CIA), to an 
insufficient allocation of supergrades and a consequent inability to retain equivalent 
personnel are also mistaken; as of 1975 the difference in the grade structures of DIA 
and CIA had been negligible. 53 And the fact that CIA analysts cannot rise to a 
supergrade position (GS-16 to 18) without becoming administrators is not convincing 
evidence that good analysts are underpaid; GS-15 salaries are higher than the 
maximum for most tenured professors. 

Non-military analysts, or high-ranking soldiers with no promotions to look 
forward to, have fewer professional crosspressures to contend with than military 
intelligence officers. But an analyst’s autonomy varies inversely with his influence, and 
hortatory injunctions to be steadfast and intellectually honest cannot ensure that he 
will be; they cannot transcend political realities or the idiosyncrasies of leaders. 
Richard Helms notes that “there is no way to insulate the DCI from unpopularity at 
the hands of Presidents or policymakers if he is making assessments which run counter 
to administrative policy. That is a built-in hazard of the job. Sensible Presidents 
understand this. On the other hand they are human too.” Integrity untinged by 
political sensitivity courts professional suicide. If the analyst insists on perpetually 
bearing bad news, he is likely to be beheaded. Helms himself succumbed to policy 
makers’ pressures in compromising estimates of the MIRV capabilities of the Soviet SS-9 
missile in 1969, and the prospects for Cambodia in 1970. 54 The same practical 
psychological constraints are reflected in an incident in which Chief of Naval 
Operations Elmo Zumwalt, who had already infuriated Nixon and Kissinger several 
times with his strategic estimates, was determined to present yet another unwelcome 
analysis; Secretary of Defense Schlesinger dissuaded him with the warning, “To give a 
briefing like that in the White House these days would be just like shooting yourself in 
the foot.” 55 

6. Cognitive rehabilitation and methodological consciousness. The intertwining 
of analysis and decision and the record of intelligence failures due to mistaken 


53 SSCI, Final Report (fn. 7), I, 352. A valid criticism is that military personnel systems and promotion 
standards penalized intelligence officers, thus encouraging competent officers to avoid intelligence 
assignments. This situation was rectified in the service intelligence agencies by the early 1970s, but not 
within DIA. Ibid.', Betts (fn. 5), 196-97. 

M SSCI, Final Report (fn. 7), I, 77-82. See also U.S., Congress, Senate, Committee on Foreign 
Relations, Hearings, National Security Act Amendment, 92d Cong., 2d sess., 1972, 14-24. 

“ Zumwalt, On Watch (New York: Quadrangle 1976), 459. 


50 


Approved for Release: 2014/09/10 C00612718 



Approved for Release: 2014/09/10 C00612718 


Analysis , War and Decision 


preconceptions and unexamined assumptions suggest the need to reform the 
intelligence consumers’ attitudes, awareness, and modes of perception. If leaders'were 
made self-conscious and self-critical about their own psychologies, they might be less 
vulnerable to cognitive pathologies. This approach to preventing intelligence failure is 
the most basic and metaphysical. If policy makers focused on the methodologies of 
competing intelligence producers, they would be more sensitive to the biases and leaps 
of faith in the analyses passed to them. “In official fact-finding . . . the problem is not 
merely to open up a wide range of policy alternatives but to create incentives for 
persistent criticism of evidentiary value.” 56 Improvement would flow from 
mechanisms that force decision makers to make explicit rather than unconscious 
choices, to exercise judgment rather than engage in automatic perception, and to 
enhance their awareness of their own preconceptions. 57 

Unlike organizational structure, however, cognition cannot be altered by 
legislation. Intelligence consumers are political men who have risen by being more 
decisive than reflective, more aggressive than introspective, and confident as much as 
cautious. Few busy activists who have achieved success by thinking the way that they 
do will change their way of thinking because some theorist tells them to. Even if they 
could be forced to confront scholarly evidence of the dynamics of misperception, it is 
uncertain that they could consistently internalize it. Preconception cannot be 
abolished; it is in one sense just another word for “model” or “paradigm” — a construct 
used to simplify reality, which any thinker needs in order to cope with complexity. 
There is a grain of truth in the otherwise pernicious maxim that an open mind is an 
empty mind. Moreover, the line between perception and judgment is very thin, and 
consumers cannot carefully scrutinize, compare, and evaluate the methodologies of 
competing analyses, for the same prosaic reason (the problem of expertise aside) that 
impedes many proposed reforms: they do not have the time to do so. Solutions that 
require principals to invest more attention than they already do are conceptually valid 
but operationally weak. Ideally, perhaps, each principal should have a Special 
Assistant for Rigor Enforcement. 

Although most notable intelligence failures occur more often at the consuming 
than the producing end, it is impractical to place the burden for correcting those faults 
on the consumers. The most realistic strategy for improvement would be to have 
intelligence professionals anticipate the cognitive barriers to decision makers’ 
utilization of their products. Ideally, the Director of Central Intelligence should have a 
theoretical temperament and personal skills in forcing unusual analyses to the 
attention of principals; he might act as George’s “custodian” of the argumentation 
process. To fulfill this function, the DCI should be not only a professional analyst and 
an intellectual (of the twelve DCI’s since 1946, only James Schlesinger met those 
criteria, and he served for only three months), but also a skilled bureaucratic 
politician. These qualifications seldom coincide. The DCI’s coordinating staff and 
National Intelligence Officers should be adept at detecting, making explicit, and 
exposing to consumers the idiosyncracies in the assessments of various agencies — the 
reasons that the focus and conclusions of the State Department’s Bureau of 
Intelligence and Research differ from those of DIA, or of naval intelligence, or of the 
CIA. For such a procedure to work, the consumers would have to favor it (as opposed 
to negotiated consensual estimates that would save them more time). There is always a 
latent tension between what facilitates timely decision and what promotes 
thoroughness and accuracy in assessment. The fact that there is no guaranteed 


56 Wilensky (fn. 11), 164. 

57 Jervis, Perception and Misperception (fn. 22), 181-87. 
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prophylaxis against intelligence failures, however, does not negate the value of 
incremental improvements. The key is to see the problem of reform as one of modest 
refinements rather than as a systematic breakthrough. 

IV. Living with Fatalism 

Organizational solutions to intelligence failure are hampered by three basic 
problems: most procedural reforms that address specific pathologies introduce or 
accent other pathologies; changes in analytic processes can never fully transcend the 
constraints of ambiguity and ambivalence; and more rationalized information systems 
cannot fully compensate for the predispositions, perceptual idiosyncrasies, and time 
constraints of political consumers. Solutions that address the psychology and analytic 
style of decision makers are limited by the difficulty of changing human thought 
processes and day-to-day habits of judgment by normative injunction. Most theorists 
have thus resigned themselves to the hope of marginal progress, “to improve the 
‘batting average’ — say from .275 to .301 — rather than to do away altogether with 
surprise.” 58 

There is some convergence in the implications of all three ways of 
conceptualizing intelligence failures. Mistakes should be expected because the 
paradoxes are not resolvable; minor improvements are possible by reorganizing to 
correct pathologies; and despair is unwarranted because, seen in perspective, the 
record could be worse. Marginal improvements have, in fact, been steadily instituted 
since World War II. Although many have indeed raised new problems, most have 
yielded a net increase in the rationalization of the system. The diversification of 
sources of estimates of adversaries’ military power has grown consistently, obviating 
the necessity to rely exclusively on military staffs. The resources and influence of 
civilian analysts of military data (principally in the CIA’s Office of Strategic Research 
but also in its Directorate of Science and Technology) are unparalleled in any other 
nation’s intelligence system. At the same time, the DCI’s mechanism for coordinating 
the activities of all agencies — the Intelligence Community Staff — has grown and 
become more diverse and representative, and less an extension of the CIA, as more 
staffers have been added from the outside. In 1972, a separate Product Review 
Division was established within the staff to appraise the “objectivity, balance, and 
responsiveness” of intelligence studies on a regular basis. It has conducted 
postmortems of intelligence failures since then (the Yom Kippur War, the Cyprus 
crisis of 1974, the Indian nuclear test, and the seizure of the Mayaguez ). 59 (Previously, 
postmortems had been conducted by the analysts who had failed, a procedure that 
hardly guaranteed objectivity.) 

Within the Pentagon, capabilities for estimates relevant to planning were 
enhanced with the establishment of an office for Net Assessment, which analyzes the 
significance of foreign capabilities in comparison with U.S. forces. (CIA, DIA, and 
NIE’s only estimate foreign capabilities.) Civilian direction of military intelligence 
was reinforced by an Assistant Secretary of Defense for Intelligence after the 1970 
recommendation of the Fitzhugh Commission, and an Under Secretary for Policy in 
1978. Experiments in improving communication between producers and consumers 
have been undertaken (such as, for example, the testing of a Defense Intelligence 


“ Knorr (fn. 1), 460. 

59 SSCI, Final Report (fn. 7), I, 276, and IV, 85; U.S., Congress, House, Committee on Appropriations, 
Hearings, Supplemental Appropriations for Fiscal Year 1977, 95th Cong., 2d sess., 1977, 515-621; 
W ashington Post, February 15, 1977, p. A6; Paul W. Blackstock, “The Intelligence Community Under the 
Nixon Administration," Armed Forces and Society, 1 (February 1975), 238. 
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Board in late 1976). The dominance of operators within the intelligence community 
has also waned — especially since the phasing out of paramilitary operations in 
Southeast Asia and the severe reductions in size and status of CIA’s covert action 
branch that began in 1973. Dysfunctions in the military communications system, 
which contributed to crises involving intelligence collection missions in the 1960s (the 
Israeli attack on the U.S.S. Liberty and North Korea’s seizure of the Pueblo) were 
alleviated (though not cured ) by new routing procedures and by instituting an 
“optimal scanning system” in the Pentagon. 60 Statistical analyses of strategic power 
have become progressively more rigorous and comprehensive; as staffs outside the 
executive branch — such as the Congressional Budget Office — have become involved 
in the process, they have also become more competitive. 61 

Few of the changes in structure and process have generated more costs than 
benefits. (Some critics believe, however, that the abolition of the Office and Board of 
National Estimates and their replacement with National Intelligence Officers was a 
net loss.) But it is difficult to prove that they have significantly reduced the incidence 
of intelligence failure. In the area of warning, for instance, new sophisticated 
coordination mechanisms have recently been introduced, and since the institution at 
the time of the 1974 Cyprus crisis of DCI “alert memoranda” — “brief notices in a 
form which cannot be overlooked” 62 — no major warning failure has occurred. But the 
period of testing is as yet too brief to demonstrate that these adaptations are more 
effective than previous procedures. In the area of operational evaluation, it is clear 
that there was greater consciousness of the limitations and cost-ineffectiveness of aerial 
bombardment during the Vietnam War than there had been in Korea, due largely to 
the assessments made by the offices of Systems Analysis and International Security 
Affairs in the Pentagon and Secretary of Defense McNamara’s utilization of CIA 
estimates and contract studies by external analytic organizations. 63 Yet this greater 
consciousness did not prevail until late in the war because it was not a consensus; Air 
Force and naval assessments of bombing effectiveness contradicted those of the critical 
civilian analysts. Nor has the elaboration and diversification of analytic resources for 
strategic estimates clearly reduced the potential for erroneous planning decisions. 
Determination of the salience and proper weight of conflicting indicators of strategic 
power and objectives or of the comparative significance of quantitative and 


"“Joseph C. Goulden, Truth is the First Casualty (Chicago: Rand McNally 1969), 101-4; Phil G. 
Goulding, Confirm or Deny (New York: Harper & Row 1970), 130-33, 269; Pueblo and EC-121 Hearings 
(fn. 27), 646-47, 665-73, 743-44, 780-82, 802-3, 865-67, 875, 880, 897-99; Pueblo and EC-121 Report (fn. 
27), 1654-56, 1662-67; Armbrister (fn. 27), 196ff, 395; U.S., Congress, House, Committee on Armed 
Services, Report, Review of Department of Defense Worldwide Communications: Phase I, 92d Cong., 1st 
sess., 1971, and Phase II, 2d sess., 1972. 

*' See, for example, James Blaker and Andrew Hamilton, Assessing the NATO/Warsaw Pact Military 
BalancS (Washington, DC.: Congressional Budget Office, December 1977). 

K SSCI, Final Report (fn. 7), I, 61; Thomas G. Belden, “Indications, Warning, and Crisis Operations,” 
International Studies Quarterly, XXI (March 1977), 192-92. 

* 3 Pentagon Papers, IV, 111-12, 115-24, 217-32. CIA critiques of bombing results began even before 
the Tonkin Gulf crisis. CIA/OCI, Current Intelligence Memorandum, “Effectiveness of T-28 Strikes in 
Laos,” June 26, 1964; CIA/DDI, Intelligence Memorandum, “Communist Reaction to Barrel Roll Missions,” 
December 29, 1964. But ambivalence remained even within the CIA, which occasionally issued more 
sanguine evaluations — e g., CIA Memorandum for National Security Council, “The Situation in Vietnam,” 
June 28, 1965 (which McGeorge Bundy called directly to the President’s attention), and CIA/OCI, 
Intelligence Memorandum, “Interdiction of Communist Infiltration Routes in Vietnam,” June 24, 1965. (All 
memoranda are in LBJL/NSF-VNCF, Vol. I, item 5, Vol. Ill, items 28, 28a, 28b, Vol. VI A, items 4, 5, 8.) 
See also Pentagon Papers, IV, 71-74. See also the opposing assessments of the CIA. the civilian analysts in 
the Pentagon, and the Joint Chiefs in NSSM-1 (the Nixon Administration’s initial review of Vietnam policy), 
reprinted in the Congressional Record, Vol. 118, part 13, 92d Cong., 2d sess.. May 10, 1972, pp. 16749-836. 
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qualitative factors is inextricable from the political debate over foreign policy: 
uncertainties always remain, leaving the individual’s visceral fears or hopes as the 
elements that tilt the balance of judgment. 

Although marginal reforms may reduce the probability of error, the unresolvable 
paradoxes and barriers to analytic and decisional accuracy will make some incidence 
of failure inevitable. Concern with intelligence failure then coincides with concern 
about how policy can hedge against the consequences of analytic inadequacy. 
Covering every hypothetical vulnerability would lead to bankruptcy, and hedging 
against one threat may aggravate a different one. The problem is thus one of priorities, 
and hedging against uncertainty is hardly easier than resolving it. Any measures that 
clarify the cost-benefit trade-offs in policy hedges are measures that mitigate the 
danger of intelligence failure. 

One reasonable rule in principle would be to survey the hypothetical outcomes 
excluded by strategic premises as improbable but not impossible, identify those that 
would be disastrous if they were to occur, and then pay the price to hedge against 
them. This is no more practicable, however, than the pure form of worst-case analysis, 
because it requires willingness to bear and inflict severe costs for dubious reasons. 
Escalation in Vietnam, after all, was a hedge against allowing China to be tempted to 
“devour" the rest of Southeast Asia. The interaction of analytic uncertainty and 
decisional prudence is a vicious circle that makes the segregation of empirical 
intelligence and normative policy an unattainable Platonic ideal. 

In the simplest situation, the intelligence system can avert policy failure by 
presenting relevant and undisputed facts to non-expert principals who might 
otherwise make decisions in ignorance. But these simple situations are not those in 
which major intelligence failures occur. Failures occur when ambiguity aggravates 
ambivalence. In these more important situations — Acheson and Clausewitz to the 
contrary — the intelligence office may perform most usefully by not offering the 
answers sought by authorities, but by offering questions, acting as a Socratic agnostic, 
nagging decision makers into awareness of the full range of uncertainty, and making 
the authorities’ calculations harder rather than easier. Sensitive leaders will reluctantly 
accept and appreciate this function. Most leaders will not; they will make mistakes, 
and will continue to bear the prime responsibility for “intelligence” failures. Two 
general values (which sound wistful in the context of the preceding fatalism) remain to 
guide the choice of marginal reforms: anything that facilitates dissent and access to 
authorities by intelligence producers, and anything that facilitates skepticism and 
scrutiny by consumers. The values are synergistically linked; one will not improve the 
use of intelligence without the other. (A third value, but one nearly impossible to 
achieve, would be anything that increases the time available to principals for reading 
and reflection.) 

Intelligence failures are not only inevitable, they are natural. Some are even 
benign (if a success would not have changed policy). Scholars cannot legitimately view 
intelligence mistakes as bizarre, because they are no more common and no less 
excusable than academic errors. They are less forgivable only because they are more 
consequential. Error in scholarship is resolved dialectically, as deceptive data are 
exposed and regnant theories are challenged, refined, and replaced by new research. If 
decision makers had but world enough and time, they could rely on this process to 
solve their intelligence problems. But the press of events precludes the luxury of 
letting theories sort themselves out over a period of years, as in academia. My survey 
of the intractability of the inadequacy of intelligence, and its inseparability from 
mistakes in decision, suggests one final conclusion that is perhaps most outrageously 
fatalistic of all: tolerance for disaster. 

The foregoing article is Unclassified. 
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